1. Field of the Invention
The present invention relates to methods of processing signal samples representative of a color video image to produce a legalised color version of the image. Furthermore, the present invention relates to apparatuses for processing signal samples representative of a color video image to produce a legalised color version of the image.
2. Description of the Prior Art
It is well known that the colors of the rainbow, which correspond to light with a range of wavelengths which is visible to the human eye, can be represented from combinations of the colors red, green and blue. For this reason color television and video images are generated by separating the red, green and blue components of the images and sampling these components at spatially separated sampling points within the image. For example, color television cameras are provided with a dichronic element which separates the colors of an image formed within a field of view of the camera into red, green and blue components. Each of the red, green and blue components of the image is sampled in two dimensions in accordance with a row-by-column de-composition of the image. Each row is sampled at regularly displaced sampling points to produce a number of samples representing the row which produces the row-by-column de-composition of the image. These sampling points are known to those skilled in the art as pixels. Each of the samples represents one of the red, green and blue components of one of the pixels which make up the image.
The color image may be re-generated from the signal samples using a color visual display unit, by separating the signal samples representing the red, green and blue components of the pixels and feeding each respectively to one of three image generators. Each of the image generators operates to reconstruct, row-by-column, a version of the image for one of the three colors of red, green or blue which are super-imposed on a color screen. By producing the red, green and blue components of each pixel at positions on the screen corresponding to the positions of the pixels from which the color image was sampled, the color image is re-generated. Since each pixel is comprised of red, green and blue components, the relative intensity of these components produces a mixture of red, green and blue light which represents the color at the corresponding point of the image. The mixture of the red, green and blue components can therefore reproduce any of the colors of the original color image, which can be any of the colors of the rainbow. A combined effect of the three image generators is therefore to reproduce a version of the color image which is representative of the color image formed within the field of view of the television camera.
Representing a color image as red, green and blue signal samples provides a facility for transmitting, recording and reproducing the color image in some way. However, in order to reduce an amount of information which must be transmitted in order to convey the color image, known television transmission techniques and video image recording techniques convert the red, green and blue signals into color difference signals, which are generally comprised of a luminance and a first and a second chrominance signal. The luminance signal is, for example, formed by combining the red, green and blue signal components of a pixel into a single component representative of the relative strength of the light in the image at the pixel location. The first of the chrominance signals is generated by forming a difference between the luminance signal and the red signal, and the second chrominance signal is formed from the difference between the luminance signal and the blue color signal.
The color difference signal format is one example of a signal format which forms a signal space in which the pixels of a color video image can be represented, but which does not directly correspond with the red, green and blue components from which the color video image was generated. As a result, not all values of the color difference signal components representing a pixel within the color difference space correspond to pixels within the signal space formed from the red, green and blue components of the color image. For example, if the luminance component is at its minimum value of zero, then any non-zero value of the two chrominance signal components will result in a signal value which does not fall within the red, green and blue color reference space. Similarly, if the luminance signal is at a maximum value which corresponds to white light, then any non-zero values of the two chrominance signals will also not fall within the red, green and blue reference space.
Any color which does not fall within the red, green and blue reference space is an illegal color. For the example of color difference signals, any combination of the three components of the color difference signals which results in a value which does not fall within the red, green and blue color reference space will be an illegal value. Such illegal color values can be produced when the color images are transmitted or processed as, for example, color difference signals. For example, video signals are often processed in this format to introduce video effects such as color wash effects. As a result, values of the three color reference space components can be produced which are illegal values within the red, green and blue reference space. If these illegal color values are displayed within a color image, colors can result which do not match with the legal parts of the image. The color visual display unit reproducing the image may hard limit the color value to a maximum value of the component which can be displayed, and the illegal pixels may be reproduced or processed in an unpredictable way.
In an article entitled xe2x80x9cLimiting of YUV Digital Video Signalsxe2x80x9d by V G Devereux from the Research Department, Engineering Division, of the British Broadcast Corporation dated December 1987, a method of converting illegal color pixels in a form of YUV color difference signals into legal color pixels with respect to the red, green and blue (RGB) color reference space is disclosed. This method changes the components of the pixels in the YUV color difference space with respect to each other in order to convert the pixel in the corresponding red, green and blue color reference space into a legal pixel.
Having regard to the above discussion, it will be appreciated that there is a general requirement to provide a method of processing color video images in order to convert reliably illegal color pixels of the images into legal color pixels.
According to the present invention, there is provided a method of processing input signal samples representative of at least part of a color video image to produce legalised signal samples representative of a legal color version of the image, the method comprising the steps of generating an over sampled version of the input signal samples by generating at least one extra signal sample for each base input signal sample, generating adjustment factors from the input signal samples which when combined with the input signal samples have an effect of converting illegal color pixels of the color video image into legal color pixels, combining the adjustment factors with the input signal samples to produce the legalised color signal samples, decimating the legalised color signal samples to produce legalised signal samples having a sampling rate corresponding to that of the base input signal samples, generating further adjustment factors in dependence upon the decimated legalised color signal samples, and combining the further adjustment factors with the decimated legalised color signal samples.
It has been discovered that illegal colors can be produced as a result of distortion caused by aliasing errors. The distortion is produced by high frequency components of the video image in the analogue domain which are outside a maximum frequency which can be represented in accordance with a sampling rate of the input signal samples. This produces distortion in the video image as a result of aliasing errors.
To provide an improvement by reducing the effect of this distortion an over-sampled version of the input signal samples is generated so that these out-of-band components in the video image appear within the in-band components of the over-sampled version of the input signal samples. This provides a further advantage in representing the analogue video signal more accurately because the sampling points of the video image at the lower sampling rate can fall at positions which do not correspond to a maximum of the video signal. The video image is then legalised in this over sampled form, by generating and applying adjustment factors to produce legalised color signal samples. However, distortion of the color video image can occur when the over sampled version of the legalised color signal samples is filtered and decimated, since filtering and decimating the legalised color signal samples involves representing the influence of a plurality of signal samples to produce a composite decimated signal sample. By performing a further legalising process, resulting from generating further adjustment factors and combining these with the decimated version of the input signal samples, any distortion caused by decimating the over sampled version of the legalised color signal samples is substantially reduced. The term decimating refers to a process in which an over sampled signal is reduced to a version with signal samples having a sampling rate corresponding to that of the original input signal samples. This may involve filtering and then dropping the extra signal samples associated with the over sampled version, and forming the decimated version from the samples at the same positions as that of the original signal samples. Decimating in this sense can cause legal color pixels to become illegal, as this can involve changing some signal samples.
An effect of applying the further adjustment factors, can be to introduce aliasing errors in the decimated legalised color signal samples as a result of an effective expansion of the bandwidth of the video signal, which can not be represented with the sampling rate of the decimated legalised color signal samples. This can cause legal color pixels or legalised color pixels to become illegal. For this reason, the method may include the step of softening the further adjustment factors before combining the softened further adjustment factors with the decimated legalised signal samples. The term soften or softening refers to a process in which the adjustment factors are adapted, changed or processed in some way to reduce distortion which the adjustment factors un-softened can produce in the legalised color signal samples. This may be performed by filtering the further adjustment factors.
As explained above, although the input signal samples which are representative of the color video image may have values with respect to a signal space which is different from the red, green and blue signal space, an example embodiment of the invention finds particular application where the input signal samples are color difference signal samples having luminance and two color difference signal components. As such, in the case where the adjustment factors are calculated and applied with reference to the red, green and blue color reference space, the step of combining the adjustment factors with the input signal samples comprises the steps of converting the input color difference signal samples into a color reference signal samples having values with respect to three orthogonal color reference axes of red, green and blue light, combining the color reference signal samples with the adjustment factors and converting the combined color reference signal samples into color difference signal samples.
Although the adjustment factors may be digital values which are added to the input signal samples in order to generate the legalised color signal samples, in a preferred embodiment, the adjustment factors are scaling factors and the step of combining the adjustment factors with the input signal samples comprises the step of multiplying the adjustment factors with the input signal samples.
Accordingly to an aspect of the present invention, there is provided an image processing apparatus according to patent claim 6. Further features and aspects of the image processing apparatus are provided in the appended claims.